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SUMMARY 

Since  its  inception,  decision  analysis  has  made 
remarkable  strides  in  developing  a sophisticated  and  flexible 
methodology  and  in  solving  a variety  of  important  problems. 
Much  of  this  work  has  been  conducted  in  the  context  of 
ARPA's  Advanced  Decision  Technology  program.  Given  this 
promising  beginning  and  wealth  of  experience,  the  time  seems 
ripe  for  decision  analysis  to  give  a hard  look  at  which  of 
its  techniques  and  assumptions  are  the  strongest  and  which 
need  buttressing.  The  goal  of  such  an  investigation  would 
be  to  elucidate  what  makes  decision  analysis  and  talented 
decision  analysts  most  successful. 


The  vehicle  for  this  discussion  is  an  analogy  drawn 

between  decision  analysis  and  the  somewhat  older  profession 

of  psychotherapy.  Both  offer  a variety  of  techniques 

designed  to  help  people  function  in  a difficult  and  uncertain 

environment;  both  developed  rapidly,  sustained  by  a coherent 

underlying  theory  and  anecdotal  evidence  of  having  helped 

some  clients.  Over  the  past  half  century,  psychotherapy  has 

faced  a series  of  crises  concerned  with  its  transformation 

from  an  art  to  a clinical  science.  These  include  validation 

of  the  effectiveness  of  various  forms  of  therapy,  validating 

elements  of  treatment  progreuns  and  assumptions  of  the 

underlying  therapy,  improving  the  clinical  skills  of  individual 

practitioners,  and  considering  the  broader  political,  social, 

ideological  and  ethical  issues  raised  by  psychotherapy.  It 

is  hoped  that  by  considering  the  issues  which  a related  - 

profession  has  identified,  the  approaches  it  has  developed  v" ;-e  Section  bf' 

to  study  those  issues,  and  the  (partial)  conclusions  it  has  b.?;  section  □, 

n ' 

reached,  we  can  speed  the  development  of  decision  analysis. 
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Among  the  results  of  this  investigation  are: 

(a)  Well  developed  methodologies  exist  for  evaluating 
the  effectiveness  of  social  interventions  (of 
which  both  decision  analysis  and  psychotherapy 

are  examples) , combined  with  pioneering  evaluations 
like  those  of  Brown  and  Watson  at  Decisions  and 
Designs,  Inc.  (DDI) . They  could  give  leverage 
to  determine  where  decision  analysis  is  most 
cost  effective. 

(b)  Substantial  progress  has  been  made  in  assessing 
the  validity  of  some  of  the  elicitation  procedures 
used  by  decision  analysts,  however,  relatively 
little  is  known  about  the  robustness  results 
particularly  whether  they  apply  in  different 
contexts  and  with  different  judges.  The 
judgmental  side  of  some  questions,  like  how 
decision  problems  are  structured,  has  barely  been 
studied . 
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(c)  Much  of  the  success  of  a decision  analysis  may 
depend  upon  the  analyst's  self-presentation, 
ability  to  get  along  with  clients,  a sensitivity 
to  clients'  unstated  desires  and  uncertainties 
and  capacity  for  instilling  confidence.  A manual 
of  advice  on  how  to  fulfill  these  functions  would 
be  useful. 


ii 


(d)  Highly  competent  analyses  can  fail  as  guides  to 
decisions  if  they  adopt  too  narrow  a definition 
of  the  decision  problem.  One  must  consider  the 
political,  organizational  and  legal  constraints 
which  may  make  a technically  feasible  course  of 
action  socially  unfeasible.  Ways  are  needed  to 
incorporate  into  analyses  the  possibility  that 
selected  courses  of  action  will  not  be  adopted 
at  all,  or  at  least  not  as  planned. 

(e)  Decision  analysts  might  usefully  consider 
following  the  example  of  psychotherapists  in 
developing  some  sort  of  association  that  would 
monitor  how  analysts  are  trained  and  how  analyses 
are  performed,  in  order  to  protect  the  profession, 
clients,  and  the  public  from  slipshod  work. 
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1.  INTRODUCTION 


Modern  psychotherapy  traces  its  roots  back  to  the  work 
of  Freud  and  his  contemporaries  and  to  their  insights  into  the 
etiology  and  treatment  of  mental  disorders.  A critical  ^ 

component  of  their  revolutionary  perspective  was  the  realization 
that  much  inappropriate  behavior  should  be  attributed  not  to  the 
(mis)behaver  but  to  the  world  in  which  he  or  she  lives.  That 
world  continually  presents  people  with  challenges , some  of  which 
cannot  be  met  adequately,  given  people's  cognitive,  emotional 
and  physical  endowment.  Although  they  vary  from  individual  to 
ir  lual  and  within  individuals  over  time,  these  limitations 
versal  enough  that  some  challenges  would  overwhelm 
y,  meaning  that  failure  to  meet  them  is  not  a matter  of 
personal  ineptitude.  Maladaptive  responses  make  it  more 
difficult  to  meet  future  challenges  by  committing  the  individual 
to  ineffective  behavior  and  to  covering  up  what  appear  to  be 
past  mistakes.  A common  feature  of  most  treatment  plans  is 
forcing  people  to  face  explicitly  the  most  troublesome  aspects 
of  their  life  predicaments.  To  make  this  possible,  it  has  been 
necessary  to  legitimatize  talking  about  things  (e.g.,  sex) 
previously  excluded  from  proper  discourse. 


Over  the  years,  psychotherapists  have  developed  a number 
of  theories  with  both  descriptive  and  prescriptive  content. 

These  purport  to  show  both  how  people  live  and  how  they  might 
live  better,  given  personal  and  situational  limits.  Although 
these  theories  give  some  order  and  direction  to  the  complex 
business  of  understanding  and  helping  people,  therapy  has  always 
been  seen  as  something  of  an  art  form  whose  success  depends  on 
the  individual  clinician's  ability  to  pull  the  right  technique 
out  of  a bag  of  tricks  and  to  apply  it  appropriately. 
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From  the  beginning,  therapists  derived  confidence  in 
their  craft  from  the  coherence  of  their  theory,  their 
observation  of  clients  who  appeared  to  have  been  helped, 
clients'  testimony  that  therapy  had  helped  them  and  the 
obvious  inadequacy  of  leaving  clients  to  their  own  devices. 
Eventually,  perhaps  after  their  cumulative  confidence  passed 
some  threshold,  many  came  to  feel  that  these  sources  of 
evidence  were  inadequate.  For  psychotherapy  to  become  a 
scientific  endeavor,  it  would  have  to  submit  to  the  evidential 
standards  of  science:  empiricism,  interobserver  comparability, 
falsif iability  and  formal  inference.  Because  serving  these 
goals  conflicts  with  the  training  and  predisposition  of  many 
therapists  and  with  the  conduct  of  therapy,  response  has  been 
quite  varied.  Some  therapists  view  impressionistic  case 
studies  as  the  only  path  to  knowledge;  others  structure  every 
treatment  as  a research  project.  Purist  practitioners  of 
psychoanalysis  and  of  behavior  modification  might  be  seen  as 
representing  the  extremes  of  this  continuum.  The  development 
of  psychotherapy  over  the  past  fifty  years  has  been  a 
reflection  of  this  conflict  over  its  status  as  a science. 

Decision  analysis  traces  its  roots  to  the  observation 
that  many  problems  would  overwhelm  even  the  most  astute 
decision  maker;  that  their  complexity  and  enormity  induce 
maladaptive  responses  and  frequent  failures;  and  that  help 
is  needed  if  people  are  to  free  themselves  from  inappropriate 
habits  and  make  better  decisions.  Treatment  is  by  an 
interactive  relationship  with  a paid  professional  whose  job 
it  is  to  make  explicit  the  complexities  of  the  decision  maker's 
situation.  One  obstacle  to  doing  so  is  the  need  to  legitimize 
talking  about  value  conflicts  and  uncertainties  often  excluded 


^ from  public  discourse.  Although  the  decision  analyst's 

; techniques,  like  those  of  the  psychotherapist,  are  developed 

j from  a theory  with  both  normative  and  descriptive  implications, 

they  still  constitute  something  of  a bag  of  tricks,  whose  use 
requires  the  judicious  application  of  clinical  judgment.  Like 
I psychotherapy,  decision  analysis  is  advocated  because  the 

j theory  is  persuasive,  because  many  clients  say  that  it  helps 

1 them,  because  many  practitioners  are  extremely  talented,  and 

r because  the  alternative  seems  to  be  to  sink  back  into  an 

I 

abyss  (seat-of-the-pants  decision  making) . 


The  great  intellectual  challenge  facing  decision 
analysis  today  is  tackling  the  problem  of  becoming  a science 
with  which  psychotherapists  have  been  wrestling  for  the  past 
half  century.  Is  decision  analysis  to  remain  a craft  with 
great  intuitive  appeal  or  will  it  subject  itself  to  the  tests 
of  interobserver  comparability,  falsif iability , etc.?  This 
is,  of  course,  not  a one-time  decision,  but  a decision  node 
at  which  the  profession  will  find  itself  indefinitely. 

Although  all  analogies  break  down  at  some  point,  I will  draw 
upon  the  experience  of  psychotherapy  in  the  belief  that  it  can 
be  instructive  to  consider  how  a related  field  has  formulated, 
grappled  with  and  partially  solved  similar  problems. 
Particularly  because  the  progress  of  psychotherapy,  or  at 
least  that  caricature  of  psychotherapy  presented  here,  has 
proved  to  be  halting  and  indirect,  examining  the  obstacles 
it  has  faced  and  its  record  in  overcoming  them  may  ease 
decision  analysis'  way  through  the  same  process. 


One  strategy  in  psychotherapy's  scientific  coming  of 
age  has  been  to  attempt  to  validate  whole  theories  or  the 
effectiveness  of  entire  treatment  programs.  A second  has  been 


1-3 


w 


to  divide  and  conquer,  validating  particular  procedures  or 
theoretical  hypotheses.  Differential  effectiveness  and  cost 
effectiveness  of  various  procedures  have  been  major  concerns. 

As  even  we 11 -conceived  treatments  may  fail  if  improperly 
implemented,  a third  focus  has  been  on  improving  the  clinical 
skills  of  individual  therapists.  A fourth  concern  has  been 
to  consider  the  broader  context  within  which  therapy  is  done: 
the  socio-psycho-economic  milieu  within  which  both  practitioner 
and  client  live  and  the  philosophical-ethical  basis  of 
therapeutic  theory  and  practice. 

These  topics  are  examined  below  both  for  decision 
analysis  and  for  psychotherapy.  Because  they  have  been  central 
to  psychotherapy  longer  than  to  decision  analysis,  the 
presentation  is  often  unbalanced.  There  is  evidence  on 
psychotherapy  and  conjecture  about  decision  analysis.  At 
times,  these  analogies  and  implications  are  pushed  beyond 
available  data  in  the  interests  of  providing  a little 
provocative  speculation.  Somewhat  better  documented 
observations  on  the  practice  of  formal  analysis  are  given  by 
Fischhoff  (in  press) . 
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2 . EVALUATION 

More  than  other  social  sciences,  psychology  draws  its 
basic  methodology  from  the  natural  sciences  (primarily 
biology) . When  confronted  by  the  question,  "Does  psychotherapy 
work?"  (Eysenck,  1952)  , a natural  response  has  been  classic 
experimental  studies.  Clients  are  randomly  assigned  to  groups, 
some  receiving  treatment,  others  placebos;  differences  between 
the  groups  are  measured  over  time.  Even  in  the  best  of 
circumstances,  implementing  this  strategy  is  far  from  trivial; 
often  it  is  impossible.  For  example,  one  may  have  no  control 
over  assignment  to  treatment  groups,  or  have  control  and  be 
ethically  restrained  from  exercising  it.  For  those  situations, 
an  extensive  methodology  of  "quasi-experimental"  designs  has 
been  developed  (e.g.,  Campbell  & Stanley,  1966;  Riecken  & 
Boruch,  1974)  , substituting  innovative  statistical  controls 
for  unattainable  experimental  control.  At  the  extreme,  time 
series  analysis  has  been  adopted  for  studying  the  effect  of 
treatment  on  a single  client  (Jones,  Vaught  & Weinrott,  in 
press) . 

The  cumulative  wisdom  derived  from  evaluating 
psychotherapy  and  other  social  interventions  points  to  a 
number  of  intricacies  particularly  relevant  to  evaluating 
decision  analysis. 

(a)  The  fact  that  practitioners  have  been  trained  in 
a method  and  claim  to  be  carrying  it  out  is  no 
guarantee  that  they  are  (Loeber  & Weismann,  1975) . 
Assessing  the  fidelity  of  implementation  is 
crucial  for  knowing  what  is  being  evaluated 
(Abt  Associates,  1976;  Kolata,  1977). 
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(b)  Many  people  who  apparently  benefit  from  treatment 
would  have  improved  anyway,  due  to  changes  in 
their  life  circumstances  or  outlook.  For  adults 
with  neurotic  disorders,  the  rate  of  "spontaneous 
remissions"  has  been  estimated  as  between  0%  and 
90%,  with  a best  guess  of  one  half  to  two  thirds 
(Lambert,  1976;  Weinrott,  1977). 

(c)  The  success  of  some  treatments  may  be  less  due  to 
their  substantive,  theory-based  message  and 
manipulations  than  to  the  atmosphere  they  create. 
Even  with  systematic  desensitization  for  phobias, 
avowedly  the  most  successful  form  of  behavior 
modification,  there  is  doubt  that  the  effects  are 
due  to  anything  more  than  adroit  staging  (Kazdin 
& Wilcoxon,  1976) . These  "non-specific  treatment 
effects"  include  suggestion,  reduced  apprehension, 
increased  self-confidence  and  heightened  attention 
to  the  problem  (Atkinson  & Carskadden,  1975) . 

(d)  Unsubstantiated  clinical  judgments  are  not  to  be 
trusted.  Even  dispassionate  clinicians  of  high 
integrity  may  see  symptom-diagnosis  correlations 
where  there  are  none  (Chapman  & Chapman,  1967)  , 
treatment  effects  where  statistical  analysis  shows 
random  fluctuations  (Jones,  Weinrott  & Vaught, 
1975)  , a record  of  past  success  which  is 
exaggerated  (Fischhoff  & Beyth,  1975;  Langer  & 
Roth,  1975)  or  proven  treatment  programs  where 
there  is  but  folklore  and  bandwagon  effects 
(Schectman,  1977).  A lesson  here  is  always  to 
keep  systematic  records  in  anticipation  of  the 
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day  when  someone  will  have  the  time,  resources, 
and  inclination  to  conduct  a retrospective 
evaluation. 

(e)  Results  can  be  biased  by  looking  only  for  the 

positive  effects  a treatment  produces  and  ignoring 
possible  detrimental  effects  (Scriven,  1972)  or 
looking  only  for  the  negative  effects  (Poulton, 
1976).  Casualty  rates  (adverse  reactions)  seem 
to  run  about  10%  for  conventional  therapy 
(Bergin,  1971)  and  have  been  estimated  from  .2% 
to  47%  for  intensive  encounter  groups  with  the 
smallest  rates  reported  by  merchandisers  of  such 
treatments  (Hartley,  Roback  & Abramowitz,  1976). 

Inclusion  of  some  evaluation  methodology  in 
psychotherapy  training  programs  is  no  guarantee  that  graduates 
ever  do  any  evaluation  or  do  it  right  when  they  do  attempt  it. 
In  articles  reviewing  research  on  the  effectiveness  of  such 
diverse  treatments  as  marathon  encounter  groups  (Kilman  & 
Sotile,  1976),  sensitivity  training  (Smith,  P. , 1975),  drug 
abuse  reduction  (Callner,  1975),  behavioral  marriage  therapy 
(Jacobson  & Martin,  1976)  and  behavior  modification  of 
juvenile  deliquents  (Davison  & Seidman,  1974)  , one  finds 
a similar  litany  of  methodological  criticisms:  lack  of  a 
control  group,  inappropriate  control  groups,  impressionistic 
statistical  analysis,  potentially  biased  data  collectors,  lack 
of  follow-up  observations,  failure  to  check  observer 
reliability,  unrepresentative  samples  or  inappropriate  outcome 
measure(s).  Not  all  studies  are  guilty  of  these  infractions, 
but  far  too  many  are. 
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The  fact  that  psychotherapy  as  a profession  attempts 
to  evaluate  itself  is  certainly  to  its  credit.  Failure  to 
develop  an  evaluation  methodology  would  have  suggested  to 
many  that  it  had  something  to  hide.  However,  poor  methodology 
map  tip  the  balance  unfairly  against  therapists  interested  in 
demonstrating  the  efficacy  of  treatments.  Sloppy  research 
tends  to  increase  error  variance  and  makes  it  hard  to  detect 
differences  between  treatment  groups  (Perloff,  Perloff  & 

Sussna,  1976) . 

Parallels  with  decision  analysis  seem  natural.  Some 
products  labeled  "decision  analysis"  really  are  not;  and  the 
craft  should  not  be  judged  by  their  performance.  It  cannot 
be  presumed  that  everyone  who  seems  to  have  done  well  after 
decision  analysis  would  have  floundered  without  it;  good 
habits,  luck  and  situational  pressures  would  have  "spontaneously" 
produced  some  good  decisions.  Decision  analysis  may  help  a 
decision  maker  simply  because  the  analyst's  deskside  manner 
helps  the  decision  maker  focus  attention  and  resources  on  the 
problem,  and  not  because  of  the  specific  techniques  and 
axiomatic  justification  in  their  armamentarium.  And  so  on. 

Watson  and  Brown  (1975a)  have  pioneered  in  developing  a 
formal  model  with  which  to  decision  analyze  decision  analyses. 
This  seems  to  be  a promising  direction  if  combined  with  the 
evaluative  methodology  produced  by  psychotherapists  and  others. 
Perhaps  foretelling  the  difficulties  awaiting  such  efforts,  in 
two  of  the  three  case  studies  chosen  by  Watson  and  Brown  (1975b) 
the  greatest  benefits  of  the  analyses  seemed  to  come  not  from 
the  decisions  they  recommended,  but  from  their  contribution  to 
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organizational  processes  (reduction  of  controversy  and 
improvement  of  communication) , considerations  left  out  of 
Watson  and  Brovm's  formal  model  for  the  sake  of  simplicity. 

It  is,  of  course,  reassuring  to  a clinician  to  hear 
clients  say  that  the  treatment  helps  them.  However,  as 
evidence,  such  claims  are  both  insufficient  and  potentially 
obscurant.  When  we  lack  concrete  proof  of  the  efficacy  of 
our  wares,  it  is  tempting  to  retreat  to  diverse,  unmeasurable 
"touchy-feely"  benefits.  This,  for  example,  has  been  the 
response  of  transcendental  meditation  advocates  in  the  face 
of  evidence  that  their  method  has  no  benefits  beyond  those  of 
sitting  still  on  a regular  basis  (Smith,  J.C.,  1975). 

As  mentioned,  a persistent  problem  in  evaluating 
treatments  is  guaranteeing  that  they  have  been  implemented  in 
accordance  with  their  designers'  intent  (Kolata,  1977).  It 
would  not  be  fair  to  detract  from  decision  analysis  on  the 
basis  of  crude,  ineffectual  analyses  done  by  poorly  trained 
individuals  or  under  severe  time  restraints.  Or  would  it? 

Most  psychotherapists  would  agree  that  a treatment  package  must 
work  "out  in  the  world"  or  there  is  little  point  to  it.  If 
only  a selected  few  can  master  the  craft  or  if  the  masters  do 
little  to  monitor  those  acting  in  the  craft's  name,  then  its 
role  as  a panacea  is  limited.  Its  role  is  also  limited  if  the 
experience  is  so  unpleasant  or  expensive  that  few  clients  ever 
get  the  full  treatment.  A program  with  a relatively  high 
drop-out  rate  but  great  success  with  those  who  complete  it 
will  not  be  highly  regarded,  particularly  when  one  considers 
that  people  who  stay  in  treatment  are  those  most  susceptible 
to  persuasive  messages  of  any  kind  (according  to  attitude 
research;  see  Bandura,  1969). 


I 


Decision  analysis  is  a tool  designed  for  situations 
with  scarcity  and  will  almost  always  be  used  in  such  situations. 
All  the  resources  (computer  time,  analyst  fees,  decision 
makers'  attention)  needed  for  a full,  proper  decision  amalysis 
will  rarely  be  available.  Indeed,  analyses  are  always  bounded 
for  the  sake  of  manageability.  Thus  in  evaluating  the  field, 
one  must  ask,  in  effect,  does  decision  analysis  degrade 
gracefully?  A little  analysis  is  obviously  not  as  good  as 
a full-blown  one,  but  will  it  be  better  than  none  at  all? 

The  answer  may  depend  on  what  happens  when  adequate  funds  are 
denied.  Does  the  analyst  do  somewhat  less  of  everything,  or 
are  some  aspects  of  the  analysis  eliminated  entirely? 

No  clear  overview  of  the  current  state  of  decision 
analysis  now  exists.  Such  an  overview  could  be  done  by 
reviewing  a random  sample  of  recent  reports  of  decision 
analysis  and  subjecting  them  to  questions  like  the  following: 

Are  the  assumptions  of  the  analysts  listed? 

Are  the  assumptions  of  the  clients  listed  (e.g.,  those 
implicit  in  the  way  the  problem  was  formulated)? 

Are  any  of  these  assumptions  tested,  or  is  supporting 
evidence  from  other  sources  cited? 

Are  probabilities  used?  If  so,  is  any  justification 
given  for  the  particular  procedure  by  which  they  are  elicited? 

Are  probabilities  or  utilities  measured  in  more  than 

one  way? 


Are  values  elicited  from  more  than  one  person? 


I 


Are  sensi  ivity  analyses  conducted,  for  probabilities, 
for  utilities,  v?  more  than  one  factor  varying  at  once? 

Are  interactions  between  impacts  considered? 

Is  more  than  one  problem  structure  used  as  a cross 

check? 


Are  possible  alternatives  given  by  the  client  or 
created  with  the  client? 

Are  gaps  in  scientific  knowledge  noted? 

Is  a bottom  line  figure  given  and,  if  so,  how  is  it 

hedged? 


Is  the  public  involved,  and  if  so,  at  what  stage? 

Is  there  any  consideration  of  political  feasibility 
or  legal  constraints? 

Is  there  any  external  criticism  of  the  report,  and  if 
so,  has  the  analysis  been  redone  in  its  light? 

Is  there  any  indication  of  when  the  analysis  should 
be  redone  to  consider  possible  changes  of  circumstance  and 
that  such  reanalyses  will  be  done? 

Is  any  attempt  made  to  evaluate  the  analysis  or  to 
indicate  how  interested  parties  might  do  so  on  their  own? 

How  much  did  the  analysis  cost? 


I 
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If  the  reviewer  has  opinions  about  the  quality  of  the 
analyses  or  the  competence  of  the  analysts,  such  judgments  can 
be  correlated  with  answers  to  the  above  questions  to  see  what 
a good  report  is  and  what  good  analysts  do  (see  Glaser  & Taylor, 

1973  for  a related  exercise  performed  on  applied  research 
projects) . 


3.  DIVIDE  AND  CONQUER 

Recognizing  the  difficulties  of  validating  entire 
treatment  programs  and  philosophies  en  masse,  many  therapists 
have  focussed  their  research  on  the  validity  of  the  theoretical 
assumptions  upon  which  treatment  programs  are  based  and  the 
effectiveness  of  their  component  techniques.  This  strategy 
promises  to  be  both  feasible  and  instructive.  Investigators 
typically  believe  that  there  is  at  least  some  truth  to  the 
[ therapy  they  are  studying;  what  they  want  to  know  is  what  its 

strengths  are  and  how  can  it  be  improved  (Azrin,  1977). 

: Enormous  amounts  of  effort  have  gone  into  validating 

theoretical  assumptions  such  as;  stable  personality  traits 
exist  (Mischel,  1977);  feedback  facilitates  learning 
(McKeachie,  1976);  psychopathology  is  related  to  unconscious 
libidinal  and  aggressive  wishes  (Silverman,  1976) ; and  self- 
awareness  is  necessary  for  improvement  in  therapy  (Schectman, 
1977). 

An  entire  industry  has  emerged  for  the  measurement  and 
correlation  of  various  personality  and  behavioral  traits. 
Goldberg  (1974)  estimated  that  over  3000  books,  chapters,  and 
journal  articles  on  "objective  diagnostic  tests  and  measures" 

I appear  yearly  in  English  alone.  The  sophisticated  methodology 

! developed  by  personality  measures  merits  reading  by  anyone 

I . measuring  anything  at  least  moderately  squishy  (Cronbach, 

Gleser,  Nanda  & Rajaratnam,  1972;  Wiggins,  1973). 

Perhaps  because  of  a tendency  for  therapists  to  believe 
that  their  program  is  good  for  whatever  ails  one,  there  seems 
to  have  been  limited  response  to  calls  to  assess  the 
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cost-effectiveness  and  differential  effectiveness  of  various 
therapeutic  devices  (Tavormina,  1974).  Several  important 
research  projects  have,  however,  shown  that  the  full  regalia 
of  treatment  programs  is  not  always  essential  to  their 
effectiveness.  The  diagnostic  role  of  trained  clinicians  can 
often  be  supplanted  by  simple  computational  formulae  (Meehl, 
1954),  and  the  therapeutic  role  can  be  assumed  by  para- 
professionals  trained  in  "helping  slcills"  (Carkhuff,  1973; 
Danish,  D'Augelli  & Brock,  1976).  Both  types  of  replacement 
can  substantially  reduce  the  cost  of  treatment  and  increase 
the  number  of  problems  treated. 

As  might  be  expected,  the  divide-and-conquer  strategy 
has  appealed  to  students  of  decision  analysis,  if  not 
necessarily  to  decision  analysts.  Slovic,  Fischhoff  and 
Lichtenstein  (1977)  reviewed  a large  number  of  studies  which 
ask  whether  people  accept  the  normative  axioms  upon  which 
decision  analysis  is  based  and  how  well  various  elicitation 
procedures  capture  people's  utilities  and  values.  We  now 
believe,  with  varying  degrees  of  confidence,  that  people  don't 
wish  to  accept  Savage's  independence  axiom  (Moskowitz,  1974; 
Slovic  & Tversky,  1974);  that  they  sometimes,  but  not  often, 
want  their  judgments  to  be  intransitive  (Tversky,  1969) ; that 
there  don't  seem  to  be  consistent  individual  differences  in 
risk-proneness  or  aversiveness  (Davidshofer , 1976;  Wright  & 
Phillips,  1976);  and  that  verbally  expressed  preferences  are 
not  always  consistent  with  those  revealed  in  people's  behavior 
(Fischhoff,  Slovic,  Lichtenstein,  Read  & Combs,  1976,  and 


references  therein) . We  still  know  little  about  questions 
like:  Are  value  and  utility  judgments  independent?  Can  we 
acceptablv  resolve  inconsistencies  in  people's  preferences 


due  to  minor  differences  in  presentation  (Lichtenstein  & 

Slovic,  1973;  Kahneman  & Tversky,  1977)?  Will  people  reply 
honestly  to  our  questions  about  their  values  and  can  we  spot 
their  lies  or  "strategic  responses"  (Brookshire,  Ives  & 

Schulze,  1976)?  Is  it  possible  for  the  decision  analyst  to 
act  as  a neutral  agent  when  eliciting  judgments?  The  subtle, 
unintentional  ways  in  which  one  can  influence  another's 
behavior  has  been  a persistent  problem  for  would-be 
practitioners  of  non-directive,  client-centered  therapy 
(Rogers,  1951). 

Regarding  the  validation  of  particular  assessment 
techniques,  we  know  quite  a lot  about  probabilities  (they  tend 
to  reflect  overconfidence,  although  context  makes  a difference; 
see  Lichtenstein,  Fischhoff  & Phillips,  1977;  Fischhoff, 

Slovic  & Lichtenstein,  in  press) , less  about  utilities  (see 
Kneppreth,  Gustafson,  Leifer  & Johnson,  1974;  Slovic, 
Lichtenstein  & Fischhoff,  1977,  pp.  20-4;  and  von  Winterfeldt, 
1975)  and  group  aggregation  of  uncertainties  and  utilities 
(Seaver,  1976)  and  next  to  nothing  about  eliciting  the  structure 
of  problems  from  decision  makers.  What  this  area  really  needs 
are  more  studies  like  Vertinsky  and  Wong  (1975)  and  Fischer 
(1976) , which  compare  different  methods  and  use  a variety  of 
evaluative  criteria,  exploiting  some  of  the  sophisticated 
methodology  designed  for  developers  of  psychological  measures. 


It  also  wouldn't  hurt  to  study  (if  only  to  lay  to  rest) 
two  threats  to  generality  discovered  by  psychometricians. 

One  is  the  fact  that  people’s  feelin-  about  a particular 
object  and  the  numbers  they  assign  to  those  feelings  can  vary 
greatly  with  arbitrary  features  of  the  elicitation  procedure. 
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like  the  order  in  which  alternatives  are  presented,  the 
heterogeneity  of  the  set  of  alternatives,  the  contrast 
established  between  the  first  two  alternatives,  whether  the 
scale  is  bounded  and  their  preconceptions  about  how  the 
numbers  are  supposed  to  be  used  (Poulton,  1968,  1977;  Ross 
& DiLollo,  1971).  The  second  threat  is  the  fact,  noted  by 
Messick  (1975) , that  it  is  not  tests  but  responses  which 
have  validities  and  reliabilities.  Thus,  the  adequacy  of 
an  elicitation  procedure  in  one  context  with  one  particular 
set  of  individuals  is  not  a guarantee  of  universal 
applicability. 

Assuming  that  all  this  research  gets  done,  we  will 
still  need  both  an  error  theory  for  decision  analysis  and 
a taxonomy  of  decision  problems  indicating  which  variant  of 
decision  analysis  and  its  component  techniques  to  use  in  each. 
The  latter  will  tell  us  how  to  make  the  best  of  what  we  have 
and  the  former  will  tell  us  whether  what  we  have  is  good 
enough . 


As  Fischer  (1976)  notes,  without  an  error  theory  we 
cannot  know  to  what  extent  violations  of  assumptions  and 
lack  of  robustness  in  responses  threaten  the  results  of  a 
decision  analysis.  Important  steps  toward  developing  such 
a theory  (or  theories)  are: 

(a)  Fischer's  (1976)  work  with  multidimensional 
utility  models, 

(b)  von  Winterfeldt  and  Edwards'  (1973)  finding  that 
with  continuous  decision  options  (e.g. , invest 

X dollars)  some  inaccuracy  in  individual 
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probability  and  utility  assessments  will  not 
produce  terribly  suboptimal  decisions; 

(c)  Lichtenstein  et.  al.'s  (1977)  demonstration  of 
how  moderate  miscalibration  in  probability 
assessment  can  substantially  reduce  expected 
utility  with  discrete  decision  options  (e.g., 
operate/don ' t operate ) ; 

(d)  von  Winterfeldt  and  Edwards'  (1975)  identification 
of  the  ease  with  which  dominated  alternatives  . 
can  be  selected  through  improper  modeling  of 

a problem; 

(e)  Aschenbrenner  and  Kasubek's  (in  press)  finding 
that  two  different,  only  partially  overlapping, 
sets  of  attributes  produced  similar  results  in 
a multiattribute  utility  analysis; 

(f)  Kastenberg,  McKone  and  Okrent's  (1976)  discovery 
of  the  extreme  sensitivity  of  risk  assessments 
to  the  (subjective)  treatment  of  outliers;  and 

(g)  Tihansky's  (1976)  finding  that  errors  in  different 
estimates  were  positively  correlated  and, 
therefore,  would  not  tend  to  cancel  one  another 
out. 

These  are  but  pieces  of  an  error  theory.  Particularly 
useful  additions  would  be  guidelines  to  the  way  in  which 
uncertainty  from  varying  sources  (people  not  knowing  what  they 
want,  people  being  affected  by  choice  of  questioning  procedure, 
people  being  confused  by  instructions,  random  error,  etc.) 


is  compounded.  Until  an  adequate  theory  is  developed,  we 
will  have  to  be  very  generous  in  performing  sensitivity 
analyses  for  errors  arising  from  judgmental  sources. 

Several  recent  guides  to  the  selection  of  decision 
analytic  procedures  have  been  derived  from  formal  properties 
of  the  decision  situation  (e.g. , Emelyanov  & Ozernoi,  1975; 
Keeney  & Raiffa,  1976;  Pearce,  1976).  Additional  efforts 
might  look  at  more  subjective  aspects  like  the  public 
visibility  of  the  issue  at  hand,  how  well  developed  people's 
values  are,  how  much  freedom  the  analyst  and  decision  maker 
have  to  construct  alternatives , and  whether  any  evaluation  of 
the  analysis  is  planned.  Such  a guide  should  tell  us,  among 
other  things:  When,  in  order  to  avoid  misplaced  precision, 
should  all  resources  be  invested  in  problem  structuring  and 
none  in  attaching  numbers?  Can  high-priced  analysts  be 
replaced  by  para-professionals?  When  is  it  advisable  to 
acknowledge  the  poorly  developed  nature  of  people's 
preferences  and  the  limits  of  their  information-processing 
abilities  and  to  sacrifice  axiomatic  rigor  for  less  demanding 
procedures  (Edwards,  1976)?  Psychological  theories  have 
been  likened  to  box  cameras  which  take  pretty  good  pictures 
because  they  require  subjects  to  be  at  a great  distance,  in 
the  sun  and  immobile  (Zuniga,  1975);  is  the  same  true  of 
decision  analysis? 
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4.  CLINICAL  SKILLS 


However  useful  diagnostic  tests  and  psychodynamic 
theories  may  be,  therapists  know  that,  in  the  last  analysis, 
they,  themselves,  are  their  own  major  tool.  Personally, 
they  must  instill  confidence  in  clients,  choose  the 
appropriate  questioning  procedures  to  elicit  sensitive 
information,  handle  crises,  understand  what  is  not  being 
said,  avoid  imposing  their  own  values  and  perceptions,  and 
cooperate  in  creating  solutions.  To  this  end,  clinical 
psychologists  undergo  3-4  years  of  supervised  practice, 
psychiatrists  spend  one  to  two  years  in  internship  (Kiesler, 
1977) , and  psychoanalysts  undergo  psychoanalysis  to  be  fully 
aware  of  how  they  see  and  interact  with  others. 


Such  training  assumes  that  the  finer  points  of  the 
craft  can  only  be  learned  in  the  clinic  of  a master;  thus, 
many  researchers  are  attempting  to  discover  just  what  it  is 
that  makes  masters  (Goldman,  1976).  It  has  been  found,  for 


example,  that  therapists'  attitude  is  a crucial  determinant 
of  their  success,  particularly  with  low-income  clients 
(Lorion,  1974),  and  that  increasing  clients'  belief  in 
the  therapeutic  approach  and  their  confidence  in  the 
therapist  can  improve  prognoses  (Atkinson  & Carskadden, 

1975;  Kazdin  & Wilcoxon,  1975).  It  has  long  been  known  that 
one  individual  can  shape  another's  responses  with  appreciative 
grunts  and  phrases  (e.g.,  Howe  & DiMattia,  1976).  More 
recently,  it  has  been  found  that  nonverbal  communication 
(posture,  facial  expressions,  etc.)  can  account  for  up  to 
20  times  as  much  variance  in  observers'  behavior  as  verbal 
communication  (Mehrabian,  1969) . The  therapist  must  understand 
these  effects  and  how  to  use  them  to  manipulate  clients  or  how 
to  avoid  them,  as  appropriate. 


Is  the  lot  of  the  decision  analyst  that  different? 

Internship,  interviewing  skills,  instilling  confidence  . . . 

Can  one  not  imagine  an  analyst  subtly  pressuring  a client 
to  change  a probability  assessment  to  a value  the  analyst 
believes  is  more  acceptable,  using  verbal  or  nonverbal  cues 
(analyst  seems  displeased;  client  thinks,  "Well,  you're  the 
expert  on  probabilities.  Maybe  what  I meant  was  . . 

Such  manipulation  may  not  be  apparent  to  either  party; 

imagine  an  analyst  and  client  "agreeing"  that  the  latter's  i 

preferences  on  different  attributes  are  really  independent, 
making  the  elicitation  procedure  considerably  less  arduous. 

I 

Slovic  and  Tversky  (1974)  showed  how  direct  pressure  may 

i 

be  used  to  induce  clients  to  accept  axiomatic  principles.  ' 

Further  possibilities  emerge  when  the  analyst  works  with  ! 

groups.  For  example,  the  fact  that  group  discussions  tend  to  1 

polarize  opinions  (Myers  & Lamm,  1975)  suggests  that  the  j 

analyst  can  exert  some  control  over  the  group's  decision  by  | 

deciding  if  and  when  the  group  should  meet.  Plott  and  Levine  j 

(1976)  demonstrated  the  extent  to  which  group  decisions  can 

be  manipulated  by  varying  the  order  in  which  issues  are  j 

considered.  | 


I 


1 
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5.  RESISTANCE 

Resistance  to  therapy  takes  many  forms,  all  threatening 
its  success.  The  client  may  reject  the  approach  because  it 
is  not  expected  to  work  (Holen  & Kinsley,  1975) , or  because 
its  procedures  (e.g.,  talking  openly  about  sensitive  matters) 
are  threatening,  or  because  it  is  too  expensive  or  because 
of  objections  to  its  underlying  philosophy  of  life  (considered 
in  the  following  section) , or  because  the  client  is  unwilling 
to  admit  that  there  is  a problem.  The  client  who  accepts  the 
approach  may  resist  its  recommendations  because  they  require 
assuming  too  much  responsibility  for  one's  own  life,  or 
because  it  seems  easier  to  remain  somewhat  ill  than  to  adopt 
the  required  new  behavior  patterns,  or  because  they  mandate 
acknowledging  one's  own  guilt,  fallibility,  desires  or 
uncertainties.  Psychoanalytic  treatment  is  often  deemed 
incomplete  if  such  resistance  is  not  encountered. 


Even  if  the  client  is  willing  and  able  to  adopt  the 
therapeutic  approach,  treatment  may  fail  because  the  reformed 
client  leaves  therapy  for  a hostile,  unaccepting  world, 
classic  failures  of  this  type  have  been  encountered  by  the 
T-group  or  organizational  development  movement  (Alderfer, 

1977),  which  tries  to  improve  communication  in  a work  setting 
by  involving  some  workers  and  managers  in  intensive  group 
experiences  stressing  openness  and  sensitivity.  All  too  often, 
however,  the  behavioral  changes  induced  by  the  pressure  of  the 
group  situation  and  the  manipulation  of  the  group  leader  | 

vanish  when  group  members  return  to  their  hierarchic  work  ! 

settings.  Often  (one  might  argue,  always)  it  is  not  the 
client  but  the  client's  world  which  is  "sick"  and  in  need 
of  help;  the  presenting  problem  is  but  an  epiphenomenon. 


A 
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Increasingly,  therapists  are  realizing  that  treatment 
should  extend  to  the  client's  family,  work  place  and  beyond, 
and  that  the  most  cost-effective  treatment  may  be  primary 
prevention  aimed  at  improving  the  mental  health  of  an  entire 
community  (Kelly,  Snowden  & Munoz,  1977). 

Important  themes  here  are  that  the  profession's  goals 
include  making  therapy  more  cognizant  of  social  realities 
and  making  society  more  cognizant  of  the  need  for  therapy 
and  its  role  in  making  therapy  work. 


People  who  need  decision  analysis  may  reject  it 
because  they  are  personally  threatened  by  having  to  face  and 
acknowledge  their  own  doubts  and  desires,  because  they  wish 
to  avoid  decision  analysis'  public  disclosure  requirement, 
because  they  feel  uncomfortable  and  incompetent  to  deal  with 
probabilities  and  multi-attribute  certainty  equivalents, 
because  they  are  afraid  to  innovate,  or  because  someone  else 
(e.g.,  the  public)  pays  most  of  the  price  for  suboptimal 
decisions  (Behn  & Vaupel,  1976;  Keeney  & Raiffa,  1972). 


Once  a decision  analysis  has  been  performed,  its 
bottom-line  recommendations  may  be  rejected  because  they 
are  viewed  as  the  output  of  numerical  mumbo-jumbo  which  has 
no  intuitive  appeal  and  cannot  be  readily  justified  to 
superiors,  subordinates,  constituents,  etc.  Recommendations 
may  also  be  resisted  by  people  who  feel  that  they  have  not 
been  involved  early  enough  and  adequately  enough  in  the 
analysis  (Skolnikoff,  1977).  They  may  even  feel,  like 
staunch  believers  in  due  process  by  law,  that  the  decision- 
making process  and  the  values  it  embodies  and  fosters  are 
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more  important  than  the  decision  produced.  Giertz  (1976) 
describes  how  residents  of  a flood  plain  developed  a 
conservation  program  through  a series  of  ^ hoc  procedures 
with  extensive  public  participation  and  in  doing  so  solved 
to  mutual  satisfaction  equity  issues  which  were  irresolvable 
in  theory.  Even  active  participants  may  reject  the 
conclusions  of  an  analysis  if  they  have  irreconcilable 
differences  with  others  involved.  Consider,  for  example, 
Mellanby's  (1972)  insistence  that  biological  criteria  be 
the  only  ones  used  in  cost-benefit  analyses,  or  the  insistence 
upon  a noncompensatory  decision  model  by  many  opponents  of 
nuclear  power  or  recombinant  DNA  research,  for  whom  any 
chance  of  incurring  the  risks  involved  is  unacceptable, 
whatever  the  benefits.  Finally,  a decision  analysis  may  be 
resisted  for  purely  political  reasons.  Any  analysis  done  in 
the  public  domain  will  probably  end  up  supporting  one  side 
in  a dispute.  To  achieve  its  ends,  the  other  side  may  fight 
hard  and  fight  dirty,  questioning  every  fact  and  assumption 
in  the  analysis  and  casting  aspersions  on  the  integrity  of 
its  analysts,  however  well  the  analysis  is  done  and  however 
much  its  conclusions  are  qualified  (Barrager,  Judd  & North, 
1976;  Creighton,  1976).  Analysts  who  believe  in  their  work 
may  face  an  uncomfortable  choice  between  orphaning  their 
proposals,  letting  their  fate  be  decided  by  the  vicissitudes 
of  political  struggle  and  public  (mis) understanding,  or 
adopting  an  advocacy  role  for  the  ancilysis  and  thereby  for 
the  recommended  alternative. 

Even  if  clients  are  willing  to  adopt  and  able  to 
understand  the  analysts'  recommendations,  those  recommendations 
may  still  be  ignored  or  distorted  when  confronted  by  the  real 
world.  The  hostile  unaccepting  reality  faced  by  preferred 
alternatives  includes  legal,  political,  institutional,  and 
human  constraints. 
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Decision  analysis  is  new,  as  are  most  of  the  problems 
to  which  it  is  applied  and  the  very  idea  of  analytic 
evaluation  of  societal  endeavors.  As  a result,  the  legal 
statutes,  regulations,  precedents  and  interpretations 
relevant  to  decision  analysis  and  the  projects  it  considers 
are  in  a state  of  flux  (Wichelman,  1976).  Surprises, 
blockages  and  inefficiencies  are  to  be  expected.  Westman 
(1977)  , for  example,  complains  that  the  legal  mandate  given 
regulators  in  the  United  States  entrusted  with  improving 
water  quality  precludes  their  adopting  the  most  cost-effective 
methods.  Often  projects  are  held  up  so  long  and  altered  so 
extensively  in  legal  and  administrative  proceedings  that  their 
accompanying  analyses  become  antiquated.  Performing 
reanalyses  in  such  cases  is,  I believe,  the  exception  rather 
than  the  rule. 

Majone  (1976)  has  argued  persuasively  that  alternatives 
are  almost  never  adopted  as  proposed,  rather,  they  are  subject 
to  continuous  negotiation  and  alteration  by  the  parties 
concerned.  Vogel  and  Nadel  (1977)  observe  that  consumer 
advocates  in  the  United  States  direct  much  of  their  effort  to 
exposes  of  the  bureaucratic  process  in  order  to  reveal 
collaboration  between  regulators  and  regulated  industries 
and  to  pressure  regulators  in  directions  they,  themselves, 
desire. 

The  diversity  of  pressures  upon  government  officials 
may  be  so  great  that  the  only  variable  predicting  whether 
analytic  solutions  will  be  adopted  may  be  the  presence  or 
absence  of  at  least  one  individual  within  government  who  is 
fanatic  about  their  being  used  (Pack  & Pack,  1977)  . Even 
"non-political"  members  of  the  public  may  prevent  alternatives 


I 

I 


5-4 


1 


from  being  adopted  as  advocated,  by  getting  upset  over  errors 
and  side-effects  (even  those  which  are  inevitable  and 
anticipatable,  Einhorn,  1977),  by  misunderstanding  the 
results  of  formal  analyses  (Lelouche,  1977)  or  by  acting 
"irrationally”  when  rational  behavior  on  their  part  is  a key 
component  of  the  adopted  plan  (e.g.,  failure  of  the  U.S. 
National  Flood  Insurance  Plan;  Kunreuther,  1976). 

What  is  to  be  done?  An  analyst  is  no  more  justified 
evaluating  an  alternative  which  is  socially  unfeasible  than 
one  which  is  technically  impossible.  Yet  feasibility  is  both 
a relative  and  mutable  thing.  An  analyst  might  append  to 
each  alternative  a discussion  of  how  it  is  likely  to  be 
waylaid  en  route  to  implementation  and  what  needs  to  be  done 
to  keep  it  maximally  intact.  The  analyst  might  also  consider 
the  ways  in  which  each  alternative  might  be  distorted  and  the 
likelihood  of  each  possible  distortion  occurring.  Each  act 
could  then  be  treated  as  a set  of  possible  events  whose 
probabilities  are  entered  into  the  analysis  (Brown,  1975). 

The  preferred  alternative  might  turn  out  to  be  one  with 
dominated  consequences,  but  a better  chance  of  being 
implemented. 

In  the  long  run,  though,  the  adaptation  should  be 
mutual,  with  society  and  its  most  active  citizens  realizing 
their  need  to  accommodate  formal  procedures.  Toward  this 
end,  the  educational  potential  of  each  analysis  should  be 
exploited.  Public  participation  should  be  viewed  as  an 
opportunity,  not  a burden.  According  to  Roessner  (1976), 
it  is  more  important  to  build  the  analytic  capacity  of 
different  government  units  than  to  guarantee  the  adoption  of 
particular,  desirable  alternatives. 
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Attempts  to  shape  and  direct  others'  lives  cannot  be 
value  neutral.  The  therapist  who  is  "only  trying  to  help" 
has  at  the  least  made  the  evaluation  that  there  is  a situation 
needing  help.  The  therapist  who  is  "only  trying  to  do  what 
is  best  for  the  client"  cannot  avoid  at  least  some  subtle 
hints  at  what  that  "best"  is.  Even  practitioners  of  client- 
centered  therapies  whose  goal  is  to  reflect  and  clarify  their 
clients'  own  thoughts  are  still  promulgating  a world  view, 
that  people  are  responsible  for  their  own  predicaments  and 
can  extricate  themselves  if  they  only  understand  themselves 
sufficiently  well.  Baumgardner  (1976)  argues  that  the  very 
search  for  lasting  solutions  to  one’s  problems  implies  that 
the  client's  universe  has  more  orderliness  than  may  be  the 
case . 


The  ideological  biases  of  many  therapeutic  interventions 
are  familiar  intellectual  topics:  the  mechanistic  image  of 
people  projected  by  behaviorism  and  its  potential  for  control 
(Stolz,  Wienckowski  & Brown,  1975),  the  ethnocentrism  of 
psychoanalysis,  the  narcissism  of  many  contemporary  therapies 
(Marin,  1975;  Lasch,  1976a) , the  general  tendency  to  treat 
clients  as  objects  rather  than  colleagues  in  therapy  (Mischel, 
1977),  and  the  fatalism  induced  by  approaches  that  induce 
people  to  accept  their  own  life  crises  as  inevitable  (Lasch, 
1976b) . 


Even  when  a therapy's  philosophical  basis  is  acceptable, 
it  may  be  resisted  because  of  ethical  problems  or  political 
bias  in  the  way  it  is  used.  Much  opposition  to  behavior 
modification  arose  from  its  use  in  institutional  settings 
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(prisons,  asylums)  in  which  free,  informed  consent  to 
treatment  by  the  patient  is  impossible.  A frequent  problem 
for  practitioners  is  who  is  the  true  client,  the  patient  or 
someone  else  (e.g.,  a hospital  administrator)  interested  in 
maintaining  order  (Stolz  et  al,  1975).  Other  therapies  have 
lost  their  credibility  because  therapists  have  become  so 
dependent  upon  government  and  the  politically  powerful  for 
their  livelihood  that  they  have  lost  the  ability  to  make 
independent  criticism  (Snow  & Newton,  1976) , others  because 
they  can  be  afforded  only  by  the  rich,  still  others  because 
they  seem  to  be  applied  mainly  to  the  poor  (Lorion,  1974; 

Zuniga,  1975) . 

What  image  of  people  underlies  decision  analysis? 

At  first  blush,  it  seems  to  be  a highly  flattering  one. 

With  proper  coaching,  people  are  capable  of  understanding 
and  expressing  what  they  know  and  what  they  want. 

Acknowledging  their  information-processing  limitations,  they 
will  allow  these  values  and  beliefs  to  be  combined 
mechanically  and  then  accept  the  indicated  course  of  action. 

There  may,  however,  be  problems  with  this  seemingly 
innocuous  perspective.  One  is  that  it  may  create  an  illusion 
of  analyzability  for  problems  that  are  unsolvable.  Because 
decision  analysis  is  oriented  to  picking  the  apparent  best 
alternative  rather  than  to  assessing  the  adequacy  of  our 
knowledge,  it  may  encourage  us  to  act  where  ignorance  dictates 
hesitation  or  continued  information  gathering.  Because  it 
asks  us  about  everything  important,  it  may  lead  us  to  believe 
that  we  have  and  should  have  beliefs  and  opinions  about 
everything.  We  may  be  forced,  for  the  sake  of  answering 
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the  analyst,  to  create  preferences  which  have  little  to  do 
with  our  actual  desires.  Ellul  (1969)  has  argued  that 
forcing  people  to  have  (necessarily  shallow)  opinions 
aboat  many  things  is  the  best  way  to  guarantee  that  they 
have  articulated  views  about  nothing.  Ignorance  about  our 
own  beliefs  may  also  have  the  effect  of  increasing  the 
weight  given  to  consequences  bearing  easily  measured  and 
justified  monetary  values. 

The  very  reasonableness  of  decision  analysis 
involves  a political-ideological  assumption,  namely,  that 
society  is  sufficiently  cohesive  and  common-goaled  that  its 
problems  can  be  resolved  by  reason  and  without  struggle. 
Although  this  "get  on  with  business"  orientation  will  be 
pleasing  to  many,  it  will  not  satisfy  all.  For  those  who 
do  not  believe  that  society  is  in  a fine-tuning  stage, 
a technique  which  fails  to  mobilize  public  consciousness  and 
involvement  has  little  to  recommend  it. 

Like  therapy,  if  decision  analysis  is  not  biased  at 
its  core,  it  can  be  biased  in  its  application.  For  example, 
most  applications  to  societal  problems  seem  to  foster  the 
transfer  of  decision-making  power  to  a technical  elite  by 
offering  little  opportunity  for  effective  citizen 
participation  (Sewell  & O'Riordan,  1976;  Lovins,  1976). 
Although  this  trend  seems  inevitable  due  to  the  highly 
technical  nature  of  the  issues  studied,  in  principle,  it 
might  be  countered  by  hiring  representative  citizens  to 
participate  in  the  analytic  process  as  consumer-advocate 
specialists  in  a particular  issue.  Such  service  might  be 
considered  a form  of  jury  duty.  Previous  applications  have 
also  tended  to  ignore  the  issue  of  equitable  distribution 


6-3 


L 


I 

♦ 


of  good  and  bad  consequences.  Although  this  is  not  a necessary 
feature  of  decision  analysis,  repeated  omission  of  equity 
considerations  and  failure  to  develop  the  methodology  needed 
to  handle  them  will  suggest  lack  of  interest,  or  even 
evasiveness,  on  the  part  of  analysts  and  those  who  hire  them. 
If,  as  some  have  argued  (Lekachman,  1976) , most  national 
economies  are  going  into  a stage  of  little  overall  growth, 
crucial  quesrions  for  societal  endeavors  in  the  future  will 
concern  the  distribution  of  existing  wealth. 

When  analytic  resources  are  limited,  the  analyst  must 
take  cues  from  someone  about  how  to  restrict  the  alternatives 
and  consequences  considered.  That  someone  is  likely  to  be  the 
one  who  commissioned  the  study.  If  commissioners  come 
consistently  from  one  sector  of  society  and  consistently 
prefer  (or  reject  out  of  hand)  particular  kinds  of  solutions 
or  consequences,  a persistent  bias  may  be  produced.  Such 
bias  would  also  include  what  issues  are  never  analyzed  and 
how  results  are  presented.  If  the  commissioners  are  public 
officials,  there  may  be  a strong  predisposition  toward 
reports  that  bury  uncertainties  and  delicate  assumptions 
either  in  sophisticated  technical  machinations  or  in  masses 
of  undigested  data  (Carter,  1975) . 

Psychotherapy's  response  to  charges  of  ideological 
bias  has  been  fairly  minimal,  with  the  most  dramatic  proposals 
within  the  profession  being  to  encourage  truth-in-packaging: 
providing  potential  clients  with  a description  of  the 
assumptions  and  procedures  of  an  approach  and  perhaps  even 
entering  into  a contractual  agreement  (Health  Research 
Group,  1975;  Schwitzgebel , 1975)  . Its  response  to  charges  of 
improprieties  in  the  way  in  which  therapy  is  conducted  has 
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been  more  extreme.  Clinical  psychologists,  for  example,  have 
organized  as  a guild  with  rigorous  standards  for  entry,  state 
and  national  licensing,  censure  mechanisms  (albeit  not  often 
used),  external  review  of  research  proposals  and  papers,  and 
a strict  code  of  ethics. 

The  most  recent  revision  of  the  code  of  the  American 
Psychological  Association  (adopted  January  30,  1977,  after 
nine  years  of  work  and  12  drafts)  contains  sections  on: 

(a)  responsibility:  openly  discuss  the  limits  of 
your  knowledge;  do  not  deliberately  mislead 
clients;  do  not  suppress  disconf irming  data; 

(b)  competence:  recognize  your  own  limitations; 

(c)  moral  and  legal  standards:  be  aware  of  how  the 
quality  of  your  work  reflects  on  your  colleagues; 
avoid  any  action  that  will  violate  or  diminish 
the  legal  or  civil  rights  of  clients  or  others; 

(d)  public  statements;  take  full  account  of  the 
limits  and  uncertainties  of  present  psychological 
knowledge  and  techniques;  your  primary  goal  is  to 
aid  the  public  in  forming  their  own  informed 
judgments,  opinions  and  choices; 

(e)  confidentiality; 

(f)  welfare  of  the  consumer:  fully  inform  consumers 
of  the  purpose  and  nature  of  evaluative  treatment, 
educational,  or  training  procedures,  give  them 
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freedom  of  choice  with  regard  to  participation, 
recognize  your  own  needs  and  powerful  position  vis  a 
vis  clients;  contribute  a portion  of  your  services 
to  unpaid  work; 

(g)  professional  standards:  when  a colleague  violates 
ethical  standards,  psychologists  who  know  first 
hand  of  such  activities  should,  if  possible, 
attempt  to  rectify  the  situation;  failing  an 
informal  solution,  bring  the  matter  to  the 
attention  of  the  appropriate  local,  state  and/or 
'national  committee  on  professional  ethics, 
standards,  and  practices; 

(h)  utilization  of . assessment  techniques;  you  have 

1 the  responsibility  to  provide  explanations  of  the 

■ nature  and  purposes  of  the  test  results  in  language 

i 

the  client  can  understand;  when  a test  is 
; published,  it  should  be  accompanied  by  a manual 

fully  describing  its  development  rationale, 
i evidence  of  validity  and  reliability, 

i 

I qualifications  to  and  indications  for  use;  and 

(i)  pursuit  of  research  activities  (American 
Psychological  Association,  1977) . 

Whether  a guild  structure  is  needed  or  appropriate  for 
decision  analysis  is  not  for  me  to  say.  Certainly,  all  that 
calls  itself  decision  analysis  does  not  glitter.  However,  the 
I-  costs  of  policing  incompetent  analysts  might  be  substantial, 

draining  the  efforts  of  qualified  analysts,  discrediting  the 
t profession  by  unrepresentative  public  quibbling,  and  raising 
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prices  through  restraint  of  trade.  Perhaps  more  modest  steps 
might  be  appropriate,  if  any  are  needed  at  all: 

(a)  setting  up  a formal  "public  interest  decision 
analysis  group"  like  that  set  up  by  the  largest 
accounting  firms  in  the  United  States  in  order 
to  "give  accounting  away"; 

(b)  insisting  that  some  fixed  amount  of  funds 

(say  10%)  in  all  analysis  contracts  be  allocated 
to  independent  external  review; 

(c)  establishing  a professional  norm  of  participating 
in  voluntary  review  networks  (Smardon  & Woodland, 
1976-7) ; 

(d)  teaching  students  to  conduct  and  document  enough 
sensitivity  analyses  to  satisfy  a report's  most 
skeptical  critics  (Chow,  1977) ; or 

(e)  adopting  informal  guidelines  like  those  proposed 
by  Fairley  (1977)  for  experts  called  upon  to 
assess  probabilities  of  rare  accidents. 

Because  it  functions  in  the  public  domain,  in  addition 
to  the  private  sector  (commercial) , decision  analysis  faces 
ethical  dilemmas  at  least  as  challenging  as  those  faced  by 
psychotherapy.  For  example,  the  American  Psychological 
Association's  ethics  committee  was  unable  to  agree  on  how 
to  revise  their  standards  regarding  confidentiality  (leaving 
them  unchanged  from  1964)  even  without  having  to  consider 
(as  the  decision  analyst  might)  the  additional  problems  of 
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what  to  do  with  proprietary  information  or  information  that 
could  cause  public  panic  if  released.  Therapists  may  find 
themselves  forced  to  treat  a delinquent  when  they  should 
be  treating  a family.  Similarly,  analysts  may  get  well  into 
a problem  before  realizing  that  the  wrong  problem  has  been 
attacked,  or  that  the  wrong  information  has  been  provided, 
or  that  they  are  being  set  up  to  produce  an  advocacy  rather 
than  an  honest  analysis.  Therapists  often  face  the  problem 
of  how  to  assure  informed  consent  by  psychologically 
incompetent  clients  (Schwitzgebel , 1975) , whereas  analysts 
are  often  asked  to  pursue  their  craft  on  behalf  of  clients, 
perhaps  a whole  society,  judged  technically  incompetent. 
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7.  CONCLUSION 

The  analyst's  job  is  extremely  difficult.  Confronting 
the  issues  raised  above  (not  all  of  which  are  new  or  unique 
to  decision  analysis)  will  make  it  even  more  difficult. 
Nonetheless,  I believe  that  efforts  to  implement  a research 
program  exploring  these  problems  would  be  well  rewarded. 

Some  of  these  issues  have  obvious  pecuniary  importance  for 
the  long-term  prosperity  of  the  field  and  its  practitioners 
(e.g.,  proving  its  effectiveness  and  buttressing  its 
foundations).  Others,  like  examining  ideological  and 
ethical  questions,  will  be  highly  stimulating  intellectually. 
Still  others,  though,  will  seem  like  exercises  in  validating 
what  common  sense  knows  to  be  true  (e.g.,  that  there  is  more 
to  decision  analysis  than  putting  on  a good  act) . However, 
even  study  of  these  issues  may  have  merit,  for  common  sense 
may  be  wrong,  may  vary  across  individuals,  and  may  be 
superficial.  Examining  the  obvious  can  help  convince  others 
that  we  are  right,  improve  our  confidence  in  (and  willingness 
to  act  upon)  our  knowledge,  and  help  us  learn  why  we  were 
right  all  along. 
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he  effectiveness  of  various  forms  of  therapy,  validating  elements 
of  treatment  programs  and  assumptions  of  the  underlying  therapy, 
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the  issues  which  a related  profession  has  identified,  the 
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(partial)  conclusions  it  has  reached,  we  can  speed  the  development 
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